Gated Boltzmann Machine for Recognition under Occlusion

Unconstrained real world environments are often full of clutter. Therefore, robustness to occlusion is vital for any artificial recognition system. Recently, the Deep Boltzmann Machine (DBM) has been shown to be good at generative modeling and recognition of visual objects. In this work we develop an extension to the DBM framework to make the system more accurate when recognizing handwritten digits under partial occlusion. The key is the introduction of additional indicator random variables which specify where in the image to ignore the occluder. The new model is still a Boltzmann machine as some extra terms are added to the DBM energy function. During inference, the model tries to figure out what are the occluder and the “object” given an occluded image. In addition, we can easily transfer the learned occluder model to other DBMs learned on different types of data, e.g. faces or objects.